Ghost-compensation for improved stereoscopic images

ABSTRACT

A method and system for reducing visual ghost artifacts in plano-stereoscopic image transmissions is provided. Such visual ghost artifacts are reduced by retrieving a non-linear image representation comprising left and right image representations using an image processor, where each of the left and right image representations have a plurality of color component sub-images. The non-linear left and right image representations are transformed into respective linear left and right image representations. A left or right image compensation signal is generated by applying the determined at least one ghosting coefficient to its associated one of the plurality of color component sub-images. A respective compensated right or left image is generated by subtracting the left or right image compensation signal from the uncompensated linear left or right image representation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and is a continuation of U.S. patentapplication Ser. No. 11/441,735 filed May 25, 2006, entitled“Ghost-compensation for improved stereoscopic projection,” which claimsthe benefit of U.S. Provisional Patent Application No. 60/685,368 filedMay 26, 2005, both of which are hereby incorporated by reference for allpurposes.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present design relates to the projection of stereoscopic images, andin particular to reducing the effects of image leakage between left eyeand right eye views, also referred to as crosstalk or “ghosting.”

2. Description of the Related Art

Stereoscopic images are created by supplying the viewer's left and righteyes with separate left and right eye images showing the same scene fromrespective left and right eye perspectives. This is known asplano-stereoscopic image display. The viewer fuses the left and righteye images and perceives a three dimensional view having a spatialdimension that extends into and out from the plane of the projectionscreen. Good quality stereoscopic images demand that the left and righteyes are presented independent images uncorrupted by any bleed-throughof the other eye's image. In other words, stereoscopic selection orchannel isolation must be complete. Stereoscopic selection can beaccomplished to perfection using isolated individual optical paths foreach eye, as in the case of a Brewster stereoscope. But when usingtemporal switching (shuttering) or polarization for image selection, theleft channel will leak to some extent into the right eye and vice versa.The effect of this leaking is referred to as ghosting or crosstalk.

Various designers have attempted to reduce crosstalk or the ghostingartifact in stereoscopic displays. Most notably, Levy, in U.S. Pat. Nos.4,266,240, 4,287,528, and 4,517,592, lays out the basic technology forsubtracting a portion of one image from the other to reduce the ghostingeffect. Levy's implementations were directed to stereoscopic televisionsystems. Ensuing solutions draw heavily on Levy's work and addrelatively small improvements.

In the motion picture realm, many degrading artifacts have been cited inthe literature as detracting from the enjoyment of the projectedplano-stereoscopic motion picture experience, including the breakdown ofconvergence and accommodation, unequal field illumination, and lack ofgeometric congruence. None of these artifacts are more important thanleakage between left eye and right eye images. Stereoscopic movies showdeep, vivid images that create a significant, realistic perception of aspatial dimension that extends into and out from the plane of theprojection screen, and this effect is most degraded by crosstalk.

Certain solutions have been proposed to address ghosting, but many ofthe proposed solutions tend to be uniform across an image or screensurface, i.e. remove the same ghosting artifacts in the same wayregardless of screen position, environment, or any other pertinentfactor.

The present design seeks to address the issue of ghosting or crosstalkin a projected plano-stereoscopic motion picture environment. It wouldbe advantageous to offer a design that enhances or improves the displayof projected plano-stereoscopic motion pictures or images by reducingthe crosstalk associated with such motion picture or image displays overdesigns previously made available.

SUMMARY OF THE INVENTION

According to a first aspect of the present design, there is provided amethod for reducing ghost images in plano-stereoscopic imagetransmissions. The method comprises establishing a plurality of expectedghosting profiles associated with a plurality of predetermined regionson a screen, and compensating for leakage in each predetermined regionof a projected left eye image and a projected right eye image byremoving an amount of ghost images leaking from the projected left eyeimage into the projected right eye image and from the projected righteye image into the projected left eye image.

According to a second aspect of the present design, there is provided asystem for reducing ghost images in plano-stereoscopic imagetransmissions. The system comprises a processor configured to receivethe quantity of ghost artifacts and compute ghost compensationquantities for left eye images and right eye images. The processor isfurther configured to remove an amount of actual image ghost artifactsleaking from a projected left eye image into a projected right eye imageand from the projected right eye image into the projected left eyeimage. The processor is configured to compute ghost compensationquantities for each of a plurality of zones, each zone corresponding toa region on a screen having an expected ghosting profile associatedtherewith.

These and other advantages of the present invention will become apparentto those skilled in the art from the following detailed description ofthe invention and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a systematic representation of a two-projector system withthe projectors independently projecting left and right images, usingpolarizers to modulate the left and right channels;

FIG. 1B shows a single projector system employing a polarizationmodulator in the projected beam to alter the polarization state;

FIG. 1C is a systematic representation of a variation of the singleprojector system with the left and right frames projected in sequenceusing active eyewear;

FIG. 2A is a systematic representation of a system using active glassessimilar to that shown in FIG. 1C, using a direct view display ormonitor;

FIG. 2B is a systematic representation of a system using polarizationmodulation and is similar to that shown in FIG. 1B;

FIGS. 3A-3E demonstrate the effect of ghosting and a process for itscompensation;

FIG. 4 shows the use of a test pattern to characterize the ghosting at agiven installation;

FIG. 5 shows a process for producing left and right eye images that arecompensated to reduce ghosting;

FIG. 6 shows embodiments for postproduction and mastering applicationswhere compensation may occur in real-time or as an off-line render;

FIG. 7 shows real-time ghost compensation provided in a theater videoserver;

FIG. 8 shows details of the embodiment of FIG. 7;

FIG. 9 shows real-time ghost compensation performed by a stand-aloneunit between the theater video server and the projector;

FIG. 10 shows details of the embodiment of FIG. 9;

FIG. 11 shows real-time ghost compensation provided in a theaterprojector;

FIG. 12 shows details of the embodiment of FIG. 11;

FIG. 13 illustrates where real-time ghost compensation provided using anadvanced computer graphics card;

FIGS. 14A and 14B show a system for automating the ghost compensationcalibration process at an installation site, and a process flow for theautomation process;

FIGS. 15A and 15B show the improvement in the head tipping rangeprovided by ghost compensation as described herein;

FIG. 16 is a flowchart overview of operation of the present design; and

FIGS. 17A-17F illustrate a segmented approach to ghost or ghost artifactcompensation.

DETAILED DESCRIPTION OF THE INVENTION

Preferred embodiments of the present design focus on large screenprojection for entertainment, scientific, and visual modelingapplications. Such projection alternates the left and right image on thesame screen area using temporal switching or polarization to select theappropriate images for each eye. In the case of temporal switching,which may be combined with polarization modulation, the displayalternately transmits left and right eye images, and an electro-opticalor similar polarization modulator is employed as part of the selectionsystem to direct the appropriate image to the appropriate eye. Themodulator is best located at the projector and used in conjunction withanalyzer glasses worn by audience members. An alternate method is to useshuttering eyewear and dispense with the polarization switchingapproach. Selection devices are synchronized with the frame or fieldoutput of the projector to ensure that the frame or field can beperceived by the appropriate eye.

In such projection systems, crosstalk results from a variety of sources,including the imperfect polarization modulation of the displayed image,a timing mismatch between polarization switching and the frame or fieldoutput of the display, the imperfect phase of the switch, allowing thewrong eye to leak through at the beginning or end of the frame,imperfect or leaking analyzers for viewing the polarized light,polarization state contamination caused by projection screendepolarization; polarization state contamination caused by airborne dustor dust on the port glass or modulator surface, and, in a linearpolarizer selection system, relatively slight rotation of the analyzerglasses.

The present design addresses these sources of crosstalk in projectionapplications through an empirical calibration process characterizing thecrosstalk specific to the projection equipment, image polarization orshuttering equipment, projection screen, viewer image selectionequipment, and environment of a given installation. This process yields“ghosting coefficients” that characterize the measured crosstalk and areused to compensate image data at the projection site to provideinstallation-specific crosstalk cancellation.

Crosstalk is a linear phenomenon that affects each part of the image tothe same proportion. Crosstalk may be color dependent in so far as theprimary colors that make up the image may have different crosstalkcharacteristics. In such cases each color may be compensatedindividually.

The present design may be applied to the class of displays in which theentire image area is addressed or displayed simultaneously. In thiscase, the predicted ghosting is uniform across the entire screen, andcharacterization of crosstalk is preferably done by making a singlemeasurement of crosstalk for each primary color to obtain a completecharacterization with a single coefficient for each primary color.Alternative embodiments may utilize displays in which the display iswritten to the screen in lines, segments, or blocks. Where the image isdisplayed in segments, the ghosting depends on the timing of the displayof the segment, related to the switching speed of the modulator orshutter and their temporal characteristics. For segmented displays,characterization may be done for each segment, or for a sample ofsegments and then interpolated for the other segments.

The present design may also be applied in systems where the level ofghosting is different in different areas across the screen. In this casethe system creates a segmented ghosting map where different ghostcoefficients are applied in different areas of the screen. This isparticularly applicable with polarized projection on silver screens,where the level of ghosting tends to be highly dependent on theprojection angle and the viewing angle of the images.

The present design benefits both linear and circular polarizationimplementations. Linear polarization has higher extinction but greaterangular dependency of the polarizer with respect to the analyzer andshows degradation when the viewer tips his head to one side, whereascircular polarization selection has lower extinction but is far moreforgiving with regard to head tipping. Using circular polarization forimage selection can exhibit low crosstalk comparable to crosstalkobtained when employing linear selection. Using linear polarization forimage selection in accordance with the present design can provide animproved head tilting range comparable to that obtained when circularpolarization selection is employed.

FIGS. 1A, 1B and 1C show drawings of typical systems for stereoscopicprojection. Aspects of each system contribute to the crosstalk asdescribed below. FIG. 1A is a two-projector system in which projectors101, 102 independently project left and right eye images modulated bycorresponding linear or circular polarizers 103, 104. One source ofghosting in this system is incomplete conservation of polarization thatallows a residual component to leak to the eye whose channel isnominally blocked. Light is reflected from a projection screen 105.Projection screen 105 preferably conserves the polarization of the lightprojected thereon. In actual practice the screen will to some extentdepolarize the incident light, resulting in further ghosting. Glasses106 that are used to analyze the polarized light are imperfect and alsocontribute to ghosting.

FIG. 1B shows a single projector system employing a polarizationmodulator 108, such as a Projector ZScreen® by StereoGraphics®, locatedin the projected beam. A stereoscopic source drives projector 107 andprovides parallel left eye and right eye channels or left and right eyechannels in sequence on a single input. However formatted, the endresult is frames projected in a sequence of left-right-left-right, andeach particular frame may be sequenced to repeat (e.g. L1, R1, L1, R1,L2, R2, L2, R2, L3, etc.). The projected beam passes through modulator108 which switches polarization states in synchrony with the frame rateof the projector. The system images the projected beam on the viewingscreen 109 and the viewer observes the screen through passive polarizedanalyzer glasses 110.

In this system, a primary cause of ghosting is imperfect polarization ofthe analyzer glasses 110. Sometimes the depolarization artifact exhibitsa color dependency, resulting in more ghosting in one color thananother. In addition, imperfect synchronization or phasing of themodulator with respect to the field rate may result in ghosting. Inliquid crystal technologies used for modulation, a switching time on theorder of hundreds of microseconds may be required for a change in state.If a field or frame is projected during this transitional period,ghosting will be introduced.

FIG. 1C shows a variation of the single projector system. The projector111 projects left and right frames in sequence to the projection screen113 as described above. No modulator is used in the optical path at theprojector and instead of switching polarization the viewer wears activeglasses 114, such as CrystalEyes® by StereoGraphics. The active glasses114 switch a liquid crystal shutter worn over each eye between atransmissive and a blocking state in synchrony with the projected leftand right eye images. Switching is controlled by a wireless sync signalemitter 112 that communicates with the glasses via a communicationmedium, such as infrared or radio frequency, to switch them in sync withthe frame output. In this implementation, ghosting results from factorssimilar to those previously described, such as the synchronization ofthe shutters with the field rate and their imperfect dynamic range.

FIG. 2A shows another system somewhat similar to that of FIG. 1C, usinga direct view display 201 instead of a projected display. The viewingscreen alternately displays left and right eye views, and the systemsynchronizes active shuttering glasses 202 by a wireless or wiredcommunication link 203 with the frame rate of the display. FIG. 2Billustrates a system similar to that of FIG. 1B using a polarizationmodulator, such as the Monitor ZScreen® by StereoGraphics, covering thedisplay. The display 204 is viewed through the modulator 205synchronized with the frame rate of the display 206. Passive polarizedglasses 207 are used to select the appropriate image for each eye.

FIGS. 3A-3E illustrate the effect of ghosting and the basic principle ofits correction. FIG. 3A shows original uncompensated left eye and righteye images that form an image pair for creating a stereoscopicperception. FIG. 3B shows the images actually perceived by the viewer'seyes when crosstalk creates ghost images during viewing of the imagepair using one of the systems as described above, or the uncompensatedviews received by the viewer's eyes. FIG. 3C shows the isolated ghostcomponents, where the isolated components are to be subtracted from eacheye's image from FIG. 3B. In other words, the right eye image of FIG. 3Bshows a left side image and a right side image, and the left side imageis the image desired to be retained/transmitted. FIG. 3C shows theisolation, in the right eye image, of the ghosted image, here the rightside image, that is to be subtracted from the right eye total image ofFIG. 3B. FIG. 3D shows compensated images, where the ghost imageperceived at each eye is subtracted from the original image to beprovided to the opposite eye. FIG. 3E shows the left eye image and righteye image actually perceived when the compensated images of FIG. 3D areprojected and viewed through the same system that originally causedperception of the images of FIG. 3B through the display of the images ofFIG. 3A.

The systems identified in FIGS. 1A, 1B, 1C, 2A and 2B introducedifferent degrees of ghosting, depending on the quality andimplementation of the components used in the system. The amount ofghosting produced by a given installation of a given system ispreferably measured empirically to characterize the unique ghostingcharacteristics of that system. As described above, measurements arepreferably made for each primary color (i.e. each individual subpixelcolor) of the projection system. In a conventional display system, thisinvolves characterizing each of the three color channels (red, green,blue) that combine to form the color image. In systems with more (orfewer) primary colors, analogous processes apply. Because the factorsthat produce ghosting are linear effects, single point per colorcharacterizations may be made to predict the ghosting of the image as awhole.

The basic process for characterizing the ghosting or crosstalk in agiven system is to use test patterns that provide a full luminance(white or a primary color) image for one eye and a zero luminance(black) image for the other eye. These images are displayed or projectedby the system in L-R-L-R sequence. While a test pattern is displayed,the amount of light passing through the left and right eye portions of apair of analyzer glasses located in a normal use position can bemeasured. The amount of light arriving at each eye location in responseto the test patterns empirically characterizes the effects of allsources of crosstalk in the optical path between the projector and theviewer's eyes.

FIGS. 4A and 4B illustrate a process for characterizing ghosting usingtest patterns as described above. FIG. 4A shows the use of a testpattern that provides full luminance to the left eye and zero luminanceto the right eye. A luminance meter 400 may be placed behind a pair ofanalyzer glasses 402 to measure the luminance at each respective eyeposition. The luminance meter 400 may be a hand-held device that has aphotosensing element for receiving light input and measuring circuitryfor measuring and displaying or outputting data characterizing theluminance received by the photosensing element. Examples include thePhoto Research PR650 and the Minolta CL-100. Since the left and righteye images of the test pattern are displayed in an alternating fashion,the luminance meter value represents an average over time of theluminance received over many projection cycles.

Using the test pattern of FIG. 4A, in an ideal system the left eye willreceive full luminance and the right eye will receive zero luminance. Inreality, the right eye will receive some light from the full luminanceleft eye image as a result of the various crosstalk factors discussedabove. Consequently the luminance at the right eye image will typicallybe a non-zero value. This is referred to herein as the leakageluminance. The luminance measured at the left eye provides a baselinefull luminance measurement to be used as described below.

FIG. 4B shows the use of a test pattern that provides zero luminance tothe left eye and full luminance to the right eye. Using the luminancemeter, measurements are made again at both the left eye and right eyepositions. In response to this test pattern, a non-zero leakageluminance is typically measured at the left eye, while a baseline fullluminance value is measured at the right eye.

As mentioned above, ghosting may be color dependent. In such cases, thefull luminance images are primary color images, and measurements asdescribed above are made for each separate primary color, a featureavailable in various photosensing devices.

While these illustrations assume that the analyzer glasses used for themeasurements are oriented in a horizontally non-tilted alignment withrespect to the projection screen, in alternative embodiments it may bedesirable to characterize the ghosting effects with the glassespositioned at a slight horizontal tilt. Such testing can yield aghosting characterization that is slightly increased compared to that ofthe non-tilted position, however the slight overcompensation that mayresult may produce a demonstrably better acceptable head tilt range asdiscussed below with respect to FIGS. 15A and 15B. Such tilting andmeasurements can enhance the viewing for persons viewing the motionpicture or images at a slightly tilted angle from the horizontal.

The foregoing assumes that a calibration procedure occurs within aspecific environment. Alternately, the system may calibrate using amodel of a specific theater or other computer simulation, or may simplymake assumptions about the proposed environment and create GCs based onexpected viewing conditions.

Once all measurements are made, a ghosting coefficient (GC) for each eyechannel may be calculated by dividing the leakage luminance by the fullluminance. The ghosting coefficients GC provide a characterization ofthe crosstalk from one eye to the other that is created by theparticular equipment used in the particular installation where themeasurements were made. Where ghosting is color dependent, a separateghosting coefficient is calculated for each primary color.

As an example, leakage luminance may be computed in each of the red,blue, and green color realms as 10, 15, and 5, respectively, with totalor full luminance values of 100, 100, 100. The GC for red (GCR) would be0.10, or 10 percent, representing 10 leakage luminance divided by 100full luminance values. Blue and green ghosting coefficients in thisexample would be a GCB of 0.15 and a GCG of 0.05.

The ghosting coefficients are used to compensate images in a manner thatreduces the inherent crosstalk of the display system throughcancellation, such that the final images perceived by the eyes exhibitreduced or imperceptible ghosting. More specifically, the ghostingcoefficients are used to calculate ghosting components of the typeillustrated in FIG. 3C, which are then subtracted from original imagesto yield compensated images as illustrated in FIG. 3D. When displayed,these images are perceived in the manner illustrated in FIG. 3E.

The design produces each compensated image using an original image and aghosting component derived from the corresponding opposite eye image ofthe image pair as follows:R _(f) =R _(i) −L _(f) *GC  (1)L _(f) =L _(i) −R _(f) *GC  (2)

where:

R_(f) is the final compensated image for the right eye;

R_(i) is the original image for the right eye;

L_(f) is the final compensated image for the left eye;

L_(i) is the original image for the left eye; and

GC is the ghosting coefficient.

Through substitution, these equations may be used to characterize theghost-compensated images in terms of the original images as follows:R _(f)=(R _(i) −L _(i) *GC)/(1−GC ²)  (3)L _(f)=(L _(i) −R _(i) *GC)/(1−GC ²)  (4)

In the case where the ghosting coefficient is small, the GC² termbecomes small, and the equations may be approximated as:R _(f) =R _(i) −L _(i) *GC  (5)L _(f) =L _(i) −R _(i) *GC  (6)

In systems that exhibit color-dependent ghosting, the system calculatescompensated sub-images for each primary color using the ghostingcoefficient corresponding to each color.

As demonstrated below with respect to FIGS. 15A and 15B, a ghostingcoefficient may be employed that is slightly larger than the coefficientmeasured using analyzer glasses that are aligned in a non-tiltedposition, as a certain amount of overcompensation can increase theacceptable head tilt range without creating perceptible negativeghosting.

Ghosting compensation is preferably implemented in digital displaysystems in which images are represented as digital data that can bemathematically operated upon to perform image processing in accordancewith the ghosting correction equations provided above. FIG. 5 shows aprocess flow for producing right and left eye images that arecompensated for the ghosting effects of a given system. The processreceives left eye image data 500 and right eye image data 501 as inputs.Most digital image representations are non-linear, using either a powerlaw (gamma) representation or a log representation, whereas the ghostingfactors operate in the linear realm. Therefore, at point 502, theprocess initially transforms the left and right eye image data byapplying a linear transformation. In general, the linear transformationis preceded by an offset and normalization of the pixel values. Forexample, in the case of video coded signals, a black level offset(usually 64 in 10 bit representation) may be subtracted from the imagereceived, followed by application of an exponential value (usuallybetween 2.2 and 2.6) to the resultant image, and then multiplication bya scaling factor to fill the usable range (bitdepth) of the processorperforming the calculations.

After linear transformation, the system computes the ghost contributionfrom each eye image at point 504 using the formulas and coefficientsdiscussed above. The ghost contribution calculated for each images isthen subtracted at point 506 from the original opposite eye image toyield compensated linear image data. The compensated linear images maybe converted back into a non-linear form by applying the inverse of thelinear transformation applied above at point 508. Application of theinverse to convert back to non-linear form involves setting the range ofrepresentation and applying the non-linear transformation and offset.The output of this processing is compensated right and left eye images510 and 511.

In implementations where the ghost compensation is integrated into adisplay device such as a projector, the display device may not berequired to put the image representation back into a non-linearrepresentation since the linear image data may be fed directly to theimage display elements of the display device. In other words, blocks 508may not be needed and the output of blocks 506 may be applied directlyto the image display elements of the display device and may bedisplayed.

In general, ghost compensation may be performed in both real-time andnon-real-time implementations. Examples of each are provided in FIG. 6,which shows a postproduction environment where stereoscopic content isfinished for viewing at other locations (e.g. cinemas). One approachillustrated in FIG. 6 takes the original, uncompensated two-view orplano-stereoscopic images from a postproduction finishing system 600 andperforms real-time ghost compensation 602 between the postproductionfinishing system 600 and the reviewing projector 604. This provides theability to review the results of ghost compensation in real-time, andallows the reviewer to experiment with various levels of compensation.

A second approach for mastering is to use a non real-time process torender the ghost compensation into the images. This system provides anoff-line processor 606 that saves a ghost compensated master 608 whichmay be supplied later for viewing. The ghost compensated master may beused for internal review or may be used as a master for producingdistribution copies of the content. In the latter case, the ghostingcoefficients used in the compensation processing are typically selectedto be an average of the estimated ghosting coefficients present invarious viewing installations, as opposed to a value optimized for aspecific installation. The real-time and off-line compensation may beimplemented either in software, firmware or hardware.

Various real-time embodiments for use in viewing installations such ascinemas are now discussed with respect to FIGS. 7-13. In suchinstallations, image data is typically supplied by a server or player toa digital projector that uses a spatial light modulator (SLM) such as adigital micromirror device (DMD) to render a projected image from theimage data. Embodiments discussed below implement real-time compensationin the image data server or in the digital projector, either by takingadvantage of the computational capabilities inherent in these devices orby augmenting those capabilities through the incorporation of additionalhardware and associated programming. Alternatively real-timecompensation may be provided by a stand-alone device that performscompensation on image data streamed from the image date server to thedigital projector. Each of these embodiments enables the measurement anduse of ghosting coefficients that are installation specific to allowcompensation to be optimized for the viewing location.

FIG. 7 illustrates an embodiment in which an uncompensated distributioncopy of a stereoscopic movie 700 is played through a theater videoserver 702 that includes a real-time ghost compensation module 704. Theghosting coefficient(s) applied by the compensation module are measuredand calculated for the specific installation to provide optimumperformance as generally described above. The system sends thecompensated image stream 706 to the projector 708 for display on theprojection screen.

FIG. 8 shows details of an implementation of the compensation module ina theater video server architecture. The uncompensated image data isobtained by the module 802 from the server memory bus as parallel lefteye and right eye image data streams 800, 801. Serializers 812, 814 ofthe server receive left and right eye output data from the compensationmodule 802 and convert the output data to serial compensated left eyeand right eye image streams 816, 818 that may be supplied to aprojector.

Although the image processing of the compensation module may beperformed by a microprocessor acting under the control of software orfirmware, such as the native processing elements of the server itself,image processing may alternately be performed by a field programmablegate array (FPGA) 804 configured to receive image data and ghostingcoefficients as inputs and to process the image data in the mannerdiscussed with respect to FIG. 5. Associated with the FPGA 804 is amemory 806 for providing a working memory space, and ghostingcoefficient registers 808 for storing the ghosting coefficients to beapplied in the compensation processing. A programming interface 810enables control of the compensation module 802. In a simpleimplementation, the programming interface may include a set of switchesmanually set to provide a binary representation of the ghostingcoefficients to be applied. The FPGA 804 may be set in a bypass mode inwhich no compensation processing is performed. However, in more robustimplementations, the programming interface may comprise a serial port ora network interface and related circuitry for receiving ghostingcoefficients as well as receiving and executing compensation modulecontrol commands.

The primary functionality provided by the hardware is the subtraction ofghosting properties from the left eye and right eye images according toEquations (1) through (6). Compensation for ghosting thus requirescalculation of the appropriate coefficients, applying the coefficientsto the existing data, and subtracting the ghosted inverse from the imageto produce the de-ghosted image. To perform this, particularly whenthree components such as red, green, and blue are employed and ghostremoval occurs for each component of every pixel. Thus the design shiftsa great deal of data in and out in a very short amount of time, andprimary processing is loading data, performing a subtraction, andtransferring the compensated data from the processor or processingdevice.

FIG. 9 illustrates an alternative embodiment in which an uncompensateddistribution copy of a stereoscopic movie 900 is supplied from a theatervideo server 902 to a stand-alone real-time ghost compensation module904. The system initially performs the calibration function, i.emeasures and calculates ghosting coefficient(s) applied by thecompensation module for the specific installation to provide optimumperformance. The system sends the compensated image stream 906 to theprojector 908 for display on the projection screen.

FIG. 10 shows details of an implementation of the stand-alonecompensation module. This module is similar to the module of FIG. 8, butalso includes deserializers 820 and 821 that convert the input left eyeand right eye image streams 800 and 801 in serial form into parallelform for processing by the compensation module 802. Data in a theatervideo server environment, as well as other vide environments, may bereceived in serial form and processing of serial data according to themethodology described cannot occur. Thus data is converted from serialto parallel for full ghost compensation processing. All of the elementsillustrated in FIG. 10 are contained within the stand-alone device ofFIG. 9.

FIG. 11 illustrates a further alternative embodiment in which anuncompensated distribution copy of a stereoscopic movie 1100 is suppliedfrom a theater video server 1102 to a digital theater projector 1104that includes a real-time ghost compensation module 1106. The ghostingcoefficient(s) applied by the compensation module are measured andcalculated for the specific installation to provide optimum performance.

FIG. 12 shows details of an implementation of the compensation module ofthe projector embodiment. The elements of the compensation module aresimilar to those of FIGS. 8 and 10. Although the image processing of thecompensation module may be performed by a microprocessor acting underthe control of software or firmware, such as the native processingelements of the projector (e.g. the resizing engine), image processingmay be performed by a field programmable gate array (FPGA) 804 that isconfigured to receive image data and ghosting coefficients as inputs andto process the image data according to the Equations presented and inthe manner discussed with respect to FIG. 5. Associated with the FPGA804 is a memory 806 for providing a working memory space, and ghostingcoefficient registers 808 for storing the ghosting coefficients to beapplied in the compensation processing. A programming interface 810enables control of the compensation module 802. In a simpleimplementation, the programming interface may consist of a set ofswitches manually set to provide a binary representation of the ghostingcoefficients to be applied. For example, if one component includes moreghosting than another in the setup, such as a great deal of red ghostingoccurs, the programming interface may enable the operator to employ moreghost compensation in the red realm than blue and green realms. Otheraspects may be altered via the programming interface, including but notlimited to processing data within a certain established time, or otherappropriate control features.

The FPGA 804 may be set in a bypass mode in which no compensationprocessing is performed. However, in more robust implementations, theprogramming interface may comprise a serial port or a network interfaceand related circuitry for receiving ghosting coefficients as well asreceiving and executing compensation module control commands. Theprogramming interface of the compensation module 802 may communicatethrough the communications subsystem of the projector, enabling thecompensation module to be addressed through a communications port of theprojector such as an Ethernet port to receive ghosting coefficients andcommands.

The compensation module of the projector embodiment obtains left eye andright eye data from deserializers 820 and 821 of the projector. Theprojector architecture typically has the capability of accepting serial(HDSDI) or DVI inputs. The linearized compensated images generated bythe compensation module may be supplied to the image rendering elementsof the projector.

In accordance with another alternative embodiment, the substantialcomputational capability of a computer graphics output card may performcompensation in real-time on image data sent from a computer to adisplay device. This embodiment uses the capability of the graphics cardto perform the numerical computations of the compensation algorithm, inreal-time, operating from content processed or played from or through aprocessing device such as a personal computer.

FIG. 13 illustrates an embodiment in which computer generated 3D imagery1300 such as a movie or a video game is generated in real-time ornon-real-time, and the output is displayed using a 3D enabled graphicscard 1302 and a display device 1304 such as a projector or astereoscopic direct view display. The 3D graphics card is programmed toperform ghost compensation on the displayed images in real-time in themanner illustrated in FIG. 5.

FIGS. 14A and 14B illustrate an embodiment of a system for calibratingthe ghosting compensation to be performed at a particular installation.For purposes of illustration, the embodiment is shown in the context ofa theater projection system in which ghosting compensation is performedby the projector, however the system may be adapted to operate inconjunction with any of the embodiments described herein.

FIG. 14A shows the elements of the automated calibration system,including a theater video server 1400 and a projector 1402 that includesa ghost compensation module 1404. Luminance meters 1406 and 1407 may bearranged with respect to a set of analyzer glasses 1408 so as to becapable of measuring luminance during projection of a test pattern. Inalternative embodiments the luminance meters may be replaced withdigital cameras or other similar devices. A computing device 1410 suchas a laptop computer receives a signal representing measured luminancefrom the luminance meter, such as through a serial port. The computingdevice 1410 is also coupled through a local area network to the videoserver 1400 and the projector 1402 to enable the computing device tosupply data and issue commands to the server 1400 and projector 1402.

The computing device executes a calibration application that automatesthe test pattern display and analysis and the setting of ghostingcoefficients described herein. FIG. 14B illustrates a high level processflow of the calibration application and its interaction with othersystem devices. Initially the calibration application sends a command tothe video server to initiate the first eye test pattern, i.e. either thepattern that provides full luminance to the left eye or the pattern thatprovides full luminance to the right eye. While it is assumed here thatthe test pattern image data is resident in the video server, inalternative embodiments the calibration application may also supply thetest pattern image data to the video server along with any commandsnecessary to cause the server to execute the test pattern. Once the testpattern is initiated, luminance signals are received from the luminancemeters. From these signals, the system takes and stores luminancereadings. The calibration application then issues a command to the videoserver to initiate the second eye test pattern, i.e. the test patternfor the eye opposite to that of the first test pattern. Luminancesignals are again received from the luminance meters and readings aretaken and stored. The process of initiating test patterns and takingreadings is repeated as necessary to obtain readings for all primarycolors of the projector or all pertinent parameters employed in theghost compensation, such as luminance/chrominance, etc.

After all readings are obtained, the calibration application computesthe ghosting coefficients of the left eye and right eye channels in themanner described above. The calibration application then sends theghosting coefficients to the compensation module in the projector alongwith any commands necessary to store the ghosting coefficients andenable compensation processing using those coefficients. Ghostingcoefficients may take any of a variety of forms appropriate for thespecific implementation, such as in an array or arrays or via a set ofdata values in a stream or listing. For example, if a region, includinga pixel, has a red GC of 0.3, the value of 0.3 and the coordinate ofthat pixel may be transmitted to the compensation module, and similarred coefficients for all regions or pixels in the image are transmitted,typically indexed by region or pixel numbers or locations. Similar GCvalues may be transmitted for green and blue in the manner discussed.

FIGS. 15A and 15B illustrate an improvement in head tilting range thatmay be achieved in a system using linearly polarized images.Uncompensated linear polarized systems allow only a small head tiltbefore unacceptable ghosting occurs. For example, in a system where thelinear polarizers have extinction of greater than 1000:1 and the screenmaintains polarization to 99%, just 3 degrees of head tilt can induceghosting of 75:1, and the ghost component becomes objectionable. Thislevel of head tilt is difficult to maintain for many viewers. Usingghost compensation, it is possible to extend this range of tilting thatresults in acceptable viewing to approximately 8 degrees.

FIG. 15A shows the extinction versus head tilt for an uncompensatedsystem. The curve shows that an extinction ratio of 75:1 is achievableonly at tilt angles of less than 3 degrees in either direction. FIG. 15Bshows the performance when moderate ghost compensation is applied. Thecurve shows the effect of overcompensating, resulting in good extinctionat zero degrees, and maintenance of at least a 75:1 extinction ratiowithin a head tilt range of 8 degrees in either direction. Note thatwhen ghosting is overcompensated, negative ghosts are produced (darkghosts). The absolute value of the ghost has been plotted for clarity ofexplanation.

Similarly, ghost compensation in accordance with embodiments disclosedprovides enhanced performance for circular polarization applications,enabling dynamic ranges comparable to those of linear polarizationsystems to be achieved.

FIG. 16 illustrates general operation of the current design. In FIG. 16,point 1601 represents calibration of the system, wherein at a particularsite implementation the methodology of measuring the ghosting for thesetup is employed as discussed above, namely measuring the ghosting inthe specific environment, modeling the environment, or employing otherghosting measurement techniques. The result of the calibration 1601 maybe termed a ghosting profile or expected ghosting profile. At point1602, the system begins processing by computing at least one ghostcoefficient or ghost artifact coefficient based on the screen segments,regions, or zones, which may be based on the results from point 1601calibration. Each ghost artifact coefficient represents ghost artifactsleaking from the left eye image into the right eye image and from theleft eye image into the right eye image. Point 1603 represents applyingat least one ghost artifact coefficient for a left eye projected imageto a right eye projected image to form a compensated right eye projectedimage, and applying at least one ghost artifact coefficient for theright eye projected image to the left eye projected image to form acompensated left eye projected image. Point 1604 signifies removing thecompensated right eye projected image from the right eye projectedimage, and removing the compensated left eye projected image from theleft eye projected image. The result is transmitted to the screen andrepresents a projected image having ghosting or ghost artifacts removedtherefrom.

In a display system, factors that create ghosting are generallydifferent in different parts of the display. Such differences aregenerally the result of differences in the angle at which light passesthrough the optical elements and the differences in angle of reflectionoff the screen. Screen composition may contribute to the artifacts orghosts perceived. In such a construction, different ghost factors arerequired to optimize the ghost image depending on ghost position on thescreen. Typically more ghosting exists at the edges and corners of theimage than in the center of the screen.

FIGS. 17A-17F illustrate the general approach to segmented ghostcorrection, wherein the screen area or a hypothetical/theoretical screenarea is divided into a plurality of segments, regions, or zones. FIG.17A shows a typical projection layout in a movie theatre environmentwith the projector 1701 perpendicular to the screen 1702. From a centralviewing point, i.e. a seat located at the centerline of the theatre, theghosting will be roughly symmetrical about the center point on thescreen. FIG. 17B shows a screen with a typical distribution of theintensity of the ghost image. These are shown in the figure as contourlines 1703 representing edges of regions or zones having equal ghostintensity. If the projector is not projecting perpendicular to thecenter of the screen, but off axis, the distribution of ghosting willshift on the screen, as shown in FIG. 17C, with the contour lines 1704shifted off center.

The optimum correction for the theatre is created by characterizing theghosting factor across the area of the screen, generally by sampling ormodeling the amount of ghosting in each part of the screen and creatinga segmented correction map. For example, if Red/Green/Blue componentsare treated separately, blue ghosting may be significant at an edge orall edges of the screen. The blue GC at an outer region or zone, towardthe edge of the image, may be 0.4, while at the center of the screenblue ghosting may not be as significant and may therefore have a smallerGC, such as 0.15. Each zone may have different GCs or may employdifferent ghosting properties depending on the particular environment.

FIG. 17D illustrates how a screen may be broken into a grid 1705 forcharacterizing the ghosting, with sample points 1706 in the grid. Thisgrid may have a small number of sample points, or a very large number ofpoints as might be captured by a digital camera or modeled by asophisticated computer model. The ghost factor map or GC map may be aset of constants or may be reduced to a mathematical equation or familyof equations that characterize the ghost factor (intensity of the ghost)against each segment, or the calibration data may be stored as a table.FIG. 17E illustrates a sample plot of the ghost correction factorsgenerated by a sampling procedure such as that illustrated in FIG. 17D,where sample points 1707 make up the graph. The appropriate factor isapplied to the corresponding area of the image. As shown in FIGS. 17Dand 17E, the points may be characterized by the row in which the pixelsreside, such as Rows A and C of FIG. 17D exhibiting the same profile inFIG. 17E. Other profiles may be realized, such as groups of rows,columns, zones, or regions having similar or identical profiles, or allprofiles may differ.

The foregoing outlines a general case where the ghost factor ispotentially different for every point on the screen. From a morepractical point of view, the correction may be applied in the horizontaldirection only such as is illustrated in FIG. 17F. FIG. 17F illustratesthe screen broken into vertical regions, zones, or strips 1708 where thesame factor is applied on each strip. The plot 1711 shows an example ofhow the ghost factor might vary across each vertical strip. The endresult is a cleaner picture viewed in the specific environment, withless ghosting apparent to viewers in the theatre.

We have described a means for improving the projection of stereoscopicmotion picture images, for a variety of uses but primarily for thetheatrical motion picture industry. The application of ghostcompensation technology allows for clearer, sharper, deeper stereoscopicmovies with better off-screen effects. Preferred embodiments usereal-time pre-compensation based on ghosting characteristics measured atthe installation site so that the compensation is tailored to thecharacteristics of the individual screening room or theatre. Anadvantage of local ghosting characterization and processing is that onlyone type of print needs to be distributed for all theatres. Thus thisprint may be used in any theater for either planar exhibition orstereoscopic exhibition. Thus, by the real-time addition of the ghostpre-compensation at the projector or server, the distributors andexhibitors enjoy the economic and logistical advantages of using asingle inventory of prints for all applications.

The circuits, devices, processes and features described herein are notexclusive of other circuits, devices, processes and features, andvariations and additions may be implemented in accordance with theparticular objectives to be achieved. For example, devices and processesas described herein may be integrated or interoperable with otherdevices and processes not described herein to provide furthercombinations of features, to operate concurrently within the samedevices, or to serve other purposes. Thus it should be understood thatthe embodiments illustrated in the figures and described above areoffered by way of example only. The invention is not limited to aparticular embodiment, but extends to various modifications,combinations, and permutations that fall within the scope of the claimsand their equivalents.

The design presented herein and the specific aspects illustrated aremeant not to be limiting, but may include alternate components whilestill incorporating the teachings and benefits of the invention. Whilethe invention has thus been described in connection with specificembodiments thereof, it will be understood that the invention is capableof further modifications. This application is intended to cover anyvariations, uses or adaptations of the invention following, in general,the principles of the invention, and including such departures from thepresent disclosure as come within known and customary practice withinthe art to which the invention pertains.

The foregoing description of specific embodiments reveals the generalnature of the disclosure sufficiently that others can, by applyingcurrent knowledge, readily modify and/or adapt the system and method forvarious applications without departing from the general concept.Therefore, such adaptations and modifications are within the meaning andrange of equivalents of the disclosed embodiments. The phraseology orterminology employed herein is for the purpose of description and not oflimitation.

What is claimed is:
 1. A method for reducing visual ghost artifacts inplano-stereoscopic image transmissions using an image processing system,wherein the image transmissions are of a certain image field and whereinthe image transmissions have a plurality of color component sub-imagesand wherein the plano-stereoscopic image transmissions originate from orpass through a device that provides ghost artifact compensation, andfurther wherein at least one ghosting coefficient has been determinedfor compensating at least one of the plurality of color componentsub-images, the method comprising: retrieving a non-linear imagerepresentation comprising left and right image representations using animage processor, each of the left and right image representations havinga plurality of color component sub-images; transforming the non-linearleft and right image representations into respective linear left andright image representations using the image processor; applying thedetermined at least one ghosting coefficient to its associated one ofthe plurality of color component sub-images to generate a left or rightimage compensation signal using the image processor; and subtracting theleft or right image compensation signal from the uncompensated linearleft or right image representation using the image processor,respectively, to form a respective compensated right or left image. 2.The method of claim 1, wherein the at least one ghosting coefficientthat is applied to the associated one of the plurality of colorcomponent sub-images remains the same throughout the entire image field.3. The method of claim 1, wherein the at least one ghosting coefficientthat is applied to the associated one of the plurality of colorcomponent sub-images is applied according to a plurality of ghostingprofiles for the compensated right or left eye image, each of suchghosting profiles being associated with one of a plurality ofpredetermined regions of the image field.
 4. The method of claim 3,wherein at least one ghosting coefficient has a different value asapplied in at least two of the plurality of ghosting profiles.
 5. Themethod of claim 1, wherein the plurality of color component sub-imagesare red, blue, and green.
 6. The method of claim 5, wherein each of theplurality of color component sub-images are compensated individually. 7.The method of claim 1, wherein the reducing of the visual ghostartifacts is done substantially in real-time with displaying aplano-stereoscopic image including the compensated right or left image.8. The method of claim 7, wherein at least the subtracting occurs withina projector.
 9. The method of claim 1, wherein the reducing of thevisual ghost artifacts is done as a part of a process of creating acompensated recording to be used fir later plano-stereoscopic imagedisplay.
 10. The method of claim 1, further comprising transforming thecompensated right or left image into a non-linear compensated right orleft image using the image processor.
 11. The method of claim 1, furthercomprising applying the determined at least one ghosting coefficient togenerate the other of the right or left image compensation signals suchthat image compensation signals are generated for both the linear loftand right image representations using the image processor andsubtracting the other of the right or left image compensation signalsfrom its respective uncompensated linear left or right imagerepresentation to form a respective other compensated left or rightimage using the image processor.
 12. A method for reducing visual ghostartifacts in plano-stereoscopic image transmissions using an imageprocessing system, wherein the image transmissions are of a certainimage field and wherein the plano-stereoscopic image transmissionsoriginate from or pass through a device that provides ghost artifactcompensation, and further wherein at least one ghosting coefficient hasbeen determined for compensating the image transmissions, the methodcomprising: retrieving non-linear image data comprising left and rightimage data using an image processor; transforming the non-linear leftand right image data into respective linear left and right image datausing the image processor; applying the determined at least one ghostingcoefficient to the transformed linear left or right image data togenerate a left or right image compensation signal using the imageprocessor; and subtracting the left or right image compensation signalfrom the uncompensated linear left or right image data using the imageprocessor, respectively, to form respective compensated right or leftimage data.
 13. The method of claim 12, wherein the at least oneghosting coefficient that is applied to the transformed linear left orright image data remains the same throughout the entire image field. 14.The method of claim 12, wherein the at least one ghosting coefficientthat is applied the transformed linear left or right image data isapplied according to a plurality of ghosting profiles for thecompensated right or left eye image data, each of such ghosting profilesbeing associated with one of a plurality of predetermined regions of theimage field.
 15. The method of claim 14, wherein at least one ghostingcoefficient has a different value as applied in at least two of theplurality of ghosting profiles.
 16. The method of claim 12, wherein thereducing of the visual ghost artifacts is done substantially inreal-time with displaying a plano-stereoscopic image including thecompensated right or left image data.
 17. The method of claim 16,wherein at least the subtracting occurs within a projector.
 18. Themethod of claim 12, wherein the reducing of the visual ghost artifactsis done as a part of a process of creating a compensated recording to beused for later plano-stereoscopic image display.
 19. The method of claim12, further comprising transforming the compensated right or left imagedata into a non-linear compensated right or left image data using theimage processor.
 20. The method of claim 12, further comprising applyingthe determined at least one ghosting coefficient to generate the otherof the right or left image compensation signals such that imagecompensation signals are generated for both the linear left and rightimage data using the image processor and subtracting the other of theright or left image compensation signals from its respectiveuncompensated linear left or right image data to form a respective othercompensated left or right image data using the image processor.